Method and apparatus for interpreting images in temporal or spatial domains

ABSTRACT

This is a method and apparatus to create and interpret images in a temporal or spatial domain, which will be helpful with any moving objects. The images are captured and stored in a database. The video images can be viewed in a panoramic fashion or dissected into individual frames or pictures to make the searching of particular objects or events easier.

BACKGROUND OF THE INVENTION

A. Field of the Invention

This relates to recording video images at a remote location and placingthem into an easily readable form at a remote facility. The method andapparatus will provide adequate information about the images both from atemporal as well as a spatial perspective.

B. Prior Art

This particular device records video images at remote locations andplaces them in easy and readable forms at different remote facilities.This system employs cameras, which can be mounted at a variety ofdifferent places. There are other systems that use video capture devicesto record scenes. The cameras themselves may have infrared or low levellight capability. Representative examples of this type of device in theprior art include Vernon, U.S. Pat. No. 7,016,518, and Ciolli, U.S. Pat.No. 6,546,119.

The Vernon device provides an infrared illuminator in a camera forcapturing an image of an automobile vehicle license plate. Thisparticular device allows the operator of the device to determine whetheror not to allow a vehicle into a secured area. The Ciolli device is anautomated traffic violation monitoring and reporting system. This deviceuses a camera system, which is coupled to a data processing system forcompiling vehicle and scene images produced by the camera system. Thisis important for law enforcement agencies.

Another device to measure the speed of a vehicle includes Kupersmit,U.S. Pat. No. 5,734,337. Again, this uses a camera and specificallydetects motion.

Other examples in the prior art, which include video survey, are Gates,U.S. Pat. No. 5,073,819, and Bishop, U.S. Pat. No. 4,589,140.

None of the prior art references incorporate all the features of thisparticular method and apparatus.

BRIEF SUMMARY OF THE INVENTION

This particular method and apparatus would be a temporal datainterpretation mechanism, which would accomplish three separateobjectives.

Firstly, the method would provide the ability to conduct a search ofobjects and events in a scene. Secondly, the method would provide theability to query geographically a particular object and lastly themethod would provide a compact, easily readable presentation. The imagesmay be captured and stored in a remote location and then viewed byindividuals in a plurality of different, geographically distantlocations.

Although the railroad industry will be highlighted with thisapplication, this method and apparatus may be used in a variety of otherapplications. In fact any event which involves moving objects would besuitable for this application. Some representative venues for this typeof application may include highways, airports, parking garages or largeshopping centers.

There are several different components to this method and apparatus. Onecomponent captures and integrates the temporal information. Anothercomponent of the method allows a particular event to be viewed apartfrom other events or dissected apart from the others. The third is theobject and event search; this is important if the time of an event isknown. Another feature of this method is the ability to read text fromobjects by incorporating optical character recognition software; thisfeature is valuable if the numbering on a specific item such as arailroad car or container is known.

All captured images in the method and apparatus may be stored forsecurity, law enforcement purposes or forensic purposes. A search of thestored information on the database can direct the operator to a certaintime and space for improved security operations.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flow chart of the various system components.

FIG. 2 is a representation of some of the system components in use.

DETAILED DESCRIPTION OF THE EMBODIMENTS

This is a method by which events can be recorded and stored into adatabase 10 for eventual later use. The events are stored both in termsof time as well as space with this method 7. Particularly in the area oftrain travel this specific technology will be welcomed but may be usedwith other different applications as well.

For purposes of illustration the example of train cars 6 will be used.The train 5 will pass through a portal 1, which is a structure to whichis mounted a plurality of cameras 2. These cameras 2 will capture videoimages of the train 5 as it passes through the portal and the camerasthat are used are light sensitive 2A and may also have infraredcapability 2B. The camera may also produce a color image 2C.

Incorporated into the method and apparatus is a method by which opticalflow is measured such as the Lucas Kanade optical flow. This technologydetermines the pixel flow of the scene and from that information certainother values can be determined, including the speed of the train. Theoptical flow technology determines only pixel flow through a scene. Itis from an analysis of the pixel flow information that the speed of thetrain can be calculated.

As the train 5 passes through the portal 1, the pixel flow is measuredand a video image of each train car 6 is captured. The time of imagecapture is also recorded. The rate of capture is dependent on the speedof the train as it passes through the portal 1 and the camera speed interms of frames per second. These images are then stored in a database10.

The cameras 2, which are mounted on the portal trusses, are designed tooperate in periods of low lighting 2A and will probably have infrared 2Bcapability as well. The camera 2 is likely to be a color 2C camera aswell. The camera 2 will capture images of the number of rail cars 6including the number of containers on the rail cars and the generalcondition of any particular container or car. All the video images arethen fed to a remote location 20 and stored in a database 10.

Software is integrated into this method that will process the pixel flowinformation and produce a linear panorama 12 of the train so that anoperator at a remote facility can view the image of the entire train inan easy to view format. For example it is much more difficult to viewindividual frames of a moving object but if those individual frames areplaced in a linear panorama 12 the view is improved and theinterpretation of the image is much better and quicker. The length ofthe panorama that is presented can be extended or shortened at therequest of the operator, if and when desired.

The method also allows the database to store individual images ofindividual 14 rail cars and assign a specific time to the capture of aspecific video image. These images can also be separated from each otherso that the operator can view a single car, if desired. This may beimportant in terms of the security of a particular rail car or thesecurity of the train in general.

The method allows the operator to search for a specific car with the aidof optical recognition software 15. Optical recognition software 15 willallow a search of text, such as the number of a car or wording that mayappear on the car of a particular container or rail car. This abilityenhances the ability of the method and apparatus to search for veryspecific items, which are known or designated by the operator.

Although the example of train travel has been highlighted in thisapplication, this method may be used in any application where movingobjects are involved such as port facilities, airports, warehouses orparking garages, to name just a few examples. The method may also be auseful law enforcement tool in terms of searching for specific items orto search an area for specific items or events.

Certain background analysis 16 is performed by the capture of the imageas well as some environmental parameters 17 that are built into thesystem. Additionally, the real car height and width 18 is estimated bythe video capture. From that information a linear panorama of the movingobject, in the case the train can be provided. This provides real timeanalysis of possible problems.

Certain information about the cars and the train itself may be stored inthe database 10 so that, if certain rated parameters are violated, anindividual at the remote facility will be alerted and corrective actionmay be taken.

1. A system for interpreting images in temporal or spatial domains for atrain, comprising: a. a train; wherein the train is comprised of aseries of train cars; wherein said train cars are of predetermineddimensions; wherein said train cars are connected together; b. cameras;wherein a plurality of cameras are positioned at predetermined locationsto capture an image of a moving object; wherein the cameras capturevideo images of passing objects; wherein the cameras capture images ofindividual train cars; c. software; wherein software captures andintegrates temporal image information from the cameras; whereindissection of the video images is provided by the software; whereinoptical recognition software is integrated into the software; whereinsaid optical recognition is performed for each train car; wherein theoptical flow of the object is measured by the software; wherein saidoptical flow measures the pixel flow in the scene; wherein environmentalparameters are included in the software; wherein background analysis isprovided by the software; d. database; wherein the video imagesincluding the time of the video capture are stored in the database;wherein a time of image capture is provided for each train car; whereinsaid time of image capture for each image is recorded in the database;e. presentation; wherein a linear panorama is provided for display;wherein individual video images may be displayed; wherein the panoramacan extend a larger time frame to detect and provide a wider view of thetrain; f. remote facility; wherein the presentation is fed to a remotefacility.
 2. The system as described in claim 1 wherein the cameras areinfrared sensitive.
 3. The system as described in claim 1 wherein thecameras are color sensitive.
 4. The system as described in claim 1wherein the cameras are color cameras.
 5. The system as described inclaim 1 wherein objects are identified and stored in the database. 6.The system as described in claim 1 wherein a plurality of remotefacilities can view the same images in real time.